DOCUMENT RESUME 



ED 406 420 


TM 026 343 


AUTHOR 


Boser, Judith A. 


TITLE 


Research on Mail Survey Methods: Potential 
Confounding of Treatment Effects Due to Followup 
Contacts . 


PUB DATE 


Nov 96 


NOTE 


15p.; Paper presented at the Annual Meeting of the 
Mid-South Educational Research Association 
(Tuscaloosa, AL, November 6~8, 1996) . 


PUB TYPE 


Information Analyses (070) — Speeches/Conference 
Papers (150) 


EDRS PRICE 


MF01/PC01 Plus Postage. 


DESCRIPTORS 

ABSTRACT 


Control Groups; *Exper imental Groups; *Followup 
Studies; Incentives; *Mail Surveys; Meta Analysis; 
^Research Methodology; Research Problems; *Respons 
Rates (Questionnaires) 

This review focuses on the effects of multiple 



contacts with respondents on studies of response rates to mail 
surveys. After screening articles on research criteria, such as the 
presence of original research and sufficient reporting detail, 28 
studies remained of the original 208 identified. The difference in 
response rate between treatment and control groups (or comparison 
groups) was calculated for each point in the process, such as initial 
and followup contacts. In 16 cases, the difference in response rate 
between treatment and control (or between treatment groups) was 
increased when a second contact was instituted, In 27 cases, however, 
it was diminished, and in 7 cases it reversed. Findings tend to be 
inconclusive, but it can be seen that followups have the potential 
for confounding the results of experiments aimed at determining the 
most effective procedures for facilitating response rates in mail 
surveys. The most consistent finding is that one followup is more 
likely to decrease the effect of the manipulation than to enhance or 
reverse it. However, the magnitude of group differences must be 
considered along with the effect of the additional contact. With the 
limited number of cases available, it is not possible to draw hard 
and fast conclusions regarding many of the independent variables 
studied or other study characteristics. The 28 studies cited in the 
review are listed. (Contains 22 references.) (SLD) 



*********************************************************************** 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 

*********************************************************************** 



Research on Mail Survey Methods: 

.AT 

Potential Confounding of Treatment Effects 
Due to Followup Contacts 



U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 

Z TIONAL RESOURCES INFORMATION 
CENTER (ERIC) 

document has been reproduced as 
received from the person or organization 
originating it. 

□ Minor changes have been made to 
improve reproduction quality. 



Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL 
HAS BEEN GRANTED BY 

IT Ai &£££& . 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



by 

Judith A. Boser 



Institute for Assessment and Evaluation 
215 Qaxton Addition 
University of Tennessee 
Knoxville, TN 37996-3400 



Paper presented at the Annual Meeting of the 
Mid-South Educational Research Association, November 6-8, 1996 
Tuscaloosa, Alabama 



Research on Mail Survey Methods: Potential Confounding of Treatment Effects 

Due to Followup Contacts 

Previous research studies (Boser, 1990; Cox, Anderson & Fulcher, 1974; Erdos & Regier, 1977; 
Etzel & Walker, 1974; Futrell & Lamb, 1981; Kephart & Bressler, 1958; Lam, Malaney & Oteri,1990; 
Longworth, 1953; Martin, Duncan & Sawyer, 1984; Myers & Haug, 1969; Nichols & Meyer, 1966; Peterson, 
1975; Watson, 1965; Wiseman, 1973) and reviews (Boser & Clark, 19%; Bruvold & Comer, 1988; Conant, 
Smart & Walker, 1990; Fox, Crask, & Kim, 1988; Green & Hutchinson, 1996; Heberlein & Baumgartner, 
1978; Kanuk & Berenson, 1975; Linsky, 1975; Yammarino, Skinner & Childers, 1991), have concluded that 
followup contacts in mail surveys almost invariably result in increasing the response rates in mail 
surveys. What has not been explored, however, is the impactof followup contacts on the effect of other 
manipulations in mail survey methods. The use of a split sample approach, where those targeted to 
receive a mail questionnaire are either systematically or randomly divided into groups (experimental and 
control, or multiple experimental conditions), to study the effect of different procedures on mail survey 
response rates is commonplace. A recent review by Boser & Clark (1996) noted that in studies of mail 
survey variables where response rates were reported for more than one wave or contact, the followup 
contact(s) in some situations had a differential effect on response rates of the experimental and control 
groups or on the treatment groups being compared. That review focused on, and was organized around, 
the treatment variables, so that studies in which response rates for multiple contacts were reported were 
not examined as a group. The purpose of this study is to focus on the effects of multiple contacts on 
studies of mail survey methods. 

Methods 

The following criteria were established for studies to be included in this review: 

1. The purpose of the study was to identify variables that facilitate response rates in mail surveys. 

2. The study documented one or more original research studies and was not just a review of 
research reported elsewhere. 

3. The study used a split sample approach or an experimentally manipulated variable while other 
procedures were held constant across groups. 

4. Individuals were randomly or systematically assigned to groups. 

5. The study was done in the United States with a target population in this country. 

6. The response rate for each treatment group was reported. 

7. The sample size was provided. 

8. The written account of the survey was obtainable through published sources and written in the 
English language. 
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9. Studies in which the only manipulation was a comparison of mail surveys with other 
procedures (phone surveys, personal interviews, disk surveys, etc.) were inadmissible because they did 
not provide ways to improve mail surveys. 

10. Surveys conducted by FAX, disk or E-Mail were not included. 

1 1 . Response rates were reported after more than one wave or contact attempt. 

The first method used to locate potential articles was through computer searches of four CD- 
ROM databases: ERIC, PSYCLit, ABI/Inform, and Sociofile. These databases represent the fields most 
likely to contain research on mail survey procedures: education, psychology, business and marketing, 
and sociology respectively. ERIC contains listings of published articles and microfiche documents from 
January 1966 to the present, while PSYCLit and Sociofile began in January 1974, ABI/Inform in January of 
1986. The searches were conducted to identify all listings that contained the term "mail survey" or "mail 
surveys" in combination with "response rate" or "response rates." Some articles were listed in more than 
one database. A total of 86 articles were found in Sociofile, 125 in ABI/Inform, 125 in PSYCLit, and 175 
in ERIC. 

Reference lists of previous reviews on this topic were examined to locate additional studies. 
Current issues of key journals that routinely publish research studies on survey methods were manually 
examined in an effort to find articles that had not yet been catalogued into the CD-ROM databases. 

Abstracts of the citings were examined, and subsequendy entire articles were reviewed to 
determine their fitness for inclusion in the study. While it could be determined on the basis of the 
abstract alone that many articles did not meet the criteria for inclusion, there were also many that were 
deleted only after the article itself had been obtained and read. Some articles contained information 
about more than one study or had more than one manipulated variable within a single study. Some 
studies had been published in more than one journal. 

After screening the abstracts and articles on the first ten criteria, 208 studies were identified. 
When the final criterion was applied, 28 studies remained. Twenty-five studies provided response rates 
after the initial mailing and after one followup contact. In addition, ten studies reported response rates 
after subsequent contacts. Four studies reported response rates after one, two, and three contact attempts, 
three studies after each of four attempts. Two studies reported response rates after initial mailing and 
after the second followup; and another reported response rates after the second, third, and fourth 
followups but not after the initial mailing. For purposes of this paper, the first or initial contact is defined 
as the first mailing that includes the questionnaire, regardless of whether or not advance notification was 
implemented. The second contact is the first followup after the mailing of the questionnaire. 

The following information was recorded for each study: study identification (author and year of 
publication), target population, independent variable, sample size, and response rates for each treatment 
group or condition at each point in time available. The difference in response rate between treatment and 
control groups (or between comparison groups) was calculated at each point in time; for example, the 
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difference (di) between the response rates of the treatment (ti) and control (ci>) groups after initial 
contact (ti - ci= di) and after two contacts (initial and one followup) (t2 - C2 = d2>. 

The effect of the followup contact (fe) on the difference between groups due to the treatment 
variable was obtained by subtracting the difference between group response rates after the second contact 
from the group difference before the second contact (di - d2 = fe). The direction of change is indicated as 
either increasing (I) the group difference, decreasing (D) the difference between groups, or reversing (R) 
the effect of the independent variable as a result of the followup contact A followup effect (fe) near zero 
indicates that response rates of treatment and control groups (or comparison treatment groups) changed 
approximately the same percentage in response to the followup contacts), thus the effect of the 
independent variable was not confounded or impacted by implementing the followup(s). 

When response rates were reported after more than one followup, the difference between groups 
and the followup effect was calculated at each stage. The followup effect was considered to be the change 
in group differences since the previous followup, not a cumulative effect of all followups thatjiad 
occurred. For example, the followup effect at d4 would reflect the change in the difference between 
groups that occurred between the third contact M3) and the fourth contact M4), not the change between 
the initial mailing (di) and after the fourth contact M4). 

Results and Discussion 

Table 1 shows the effect of one followup (second contact) on treatment effects for those studies 
and conditions for which this information was available. In 16 cases, the difference in response rate 
between treatment and control (or between treatment groups) was increased when a second contact was 
instituted; in 27 cases, however, it was diminished; and in seven cases it was reversed. A number of 
variables were investigated. The variables appearing most frequently were incentives and advance 
contact. 

While there was no consistency of findings regarding the impact of a second contact when the 
independent variable was advance notification, the treatment effect of incentives was reduced when a 
followup contact was made under all but two conditions, a promised incentive of $50 and a $20 check 
(James & Bolstein, 1992). Although response rates of non-incentive groups were usually improved more 
by a second contact than were incentive groups, the response rates still clearly favor the incentive groups 
after the second contact. In one of the two cases in which the difference between groups increased, the 
amount of change was less than 1%, leaving a difference of 34%. In the other case, there was little 
difference between groups originally (2.6%) or after the second contact (6.6%). 

The amount of change after a followup contact ranged from less than one percent to a high of 36 
percent. In over one third of the cases reported, there was less than 3% change in the difference between 
group response rates as a result of a second contact effort (see Table 2). 
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Table 1 



Effect of One Followup On Treatment Effects 



Variable - Study Sample N di d 2 fe Condition 



Advance Notification 
Erdos & Regier, 1977 
Furse et al, 1981 
Kamins, 1989 

Murphy, Daley & Dalenberg, 
1991 

Smith & Hewett, 1972 
Spry et al. f 1989 

Incentives 

Erdos & Regier, 1977 

Hopkins, Hopkins, & Schon, 
1988 

James & Bolstein, 1990 
James & Bolstein, 1992 



Spry, Hovell, Sallis, Hofstetter, 
Elder, & Molgaard, 1989 

Tedin & Hofstetter, 1982 

Zusman & Duby, 1987 

Postage 

Corcoran, 1985 

ElkindTryon & deVito, 1986 

Tedin & Hofstetter, 1982 



Wall St J subscribers 


1,200 


phone customers 


907 


general public 


505 


freight forwarders 


481 


general public 


1,655 


general public 


600 


Wall St J subscribers 


800 


librarians 




cable subscribers 


850 


subcontractors 


1,200 



general public 600 

general public 1,342 

college dropouts 371 

social workers 300 

psychologists 500 

general public 1,342 



+ 4.9% 


+ 4.5% 


0.4%D 


+ 1.0% 


+ 9.0% 


8.0%I 


+ 27.8% 


+ 30.6% 


2.8%I 


+ 16.1% 


+ 19.2% 


3.1%I 


+ 12.0% 


+11.0% 


1.0%D 


+ 8.6% 


+ 7.5% 


1.1 %D 


+ 5.8% 


+ 8.1% 


23 %I 


+ 10.0% 


+ 11.3% 


13%I 


+ 8.6% 


+ 7.4% 


1.2%D 


+ 4.5% 


+ 5.4% 


0.9 %I 



+ 13.7% 


+ 8.8% 


4.9%D 


+ 21.0% 


+12.0% 


9.0%D 



+ 83% 


+ 2.6% 


5.9%D 


+ 8.9% 


+ 6.0% 


2.9%D 


+ 18.6% 


+16.2% 


2.4%D 


+ 23.4% 


+16.2% 


7.2%D 


+ 20.0% 


+ 153% 


4.7%D 


+ 28.0% 


+ 24.0% 


4.0%D 


+ 313% 


+ 26.0% 


53%D 


+ 233% 


+ 20.0% 


33%D 


+ 333% 


+ 34.0% 


0.7%I 


+ 333% 


+ 26.6% 


6.7%D 


+ 2.6% 


+ 6.6% 


4.0%I 


+ 4.6% 


+ 03% 


43%D 



+ 15.0% 


+10.0% 


5.0%D 


+ 323% 


+18.9% 


13.6%D 



+ 113% 


+ 4% 


73%D 


+ 33% 


-0.4% 


3.6%R 


+ 12.0%' 


+ 15.0% 


3.0%I 



postcard 

phone 

labeled probe foot 
probe foot 
simple foot 
solicitation group 

postcard 



letter advance contact 

phone 

postcard 



$1 in first mail (vs .25) 
$1 



.25 

.50 

$1 

$2 

$1 

$5 cash 
$5 check 
$10 check _ 
$20 check M 
$40 check ^ 
promise of $50 

lottery 



.25 

$1 



first class stamp (return) 
first class stamp (return) 
certified mail (vs first class) 
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Table 1. Effect of One Followup On Treatment Effects (continued) 



Variable - Study 


Sample 


N 


dl 


d2 


fe 


Condition 


Personalization 














Andreason, 1970 


lottery winners 


15 


- 10.0% 


-3.7% 


63%D 


typed, pers. salutation (vs mimeo) 








-3.5% 


- 2.7% 


0.8%D 


typed, pers salutation, postscript (vs 
mimeo) 








+ 6.5% 


+ 1.0% 


53%D 


postscript added 


Moore, 1941 


superintendents 


494 


+ 9.5% 


+ 163% 


7.9%I 


typed (vs duplicated) 


Roberts, McCrory & 
Forthofer, 1978 


dentists 


1,190 


+ 1.4% 


-0.9% 


23%R 


personal salutation 


Questionnaire Format 














Boser, 1990 


alumni 


300 


+ 5.6% 


+ 5.9% 


0.3 %I 


booklet (vs/f^ped 8 1/2x11) 


Boser, 1990 


alumni 


297 


+ 2.4% 


+ 23% 


0.1%I 


booklet 


Champion & Sear, 1969 


general public 


2,290 


- 10.9% 
-11.9% 
- 1.0% 


- 4.6% 
-3.1% 
+ 13% 


63%D 

8.8%D 

2.5%R 


3 pages (vs 6) 
3 pages (vs 9) 
6 pages (vs 9) 


Jacobs, 1986 


teachers 


200 


-2.0% 


+ 2.0% 


4.0%R 


use of optical scan sheets 


Qpjstlpnnrirgl^ngtlii 














Brown, 1965 


physicians 


523 


+ 15.0% 


+ 1.0% 


14.0%D 


short (postcard, 2q) vs 1 page 


Jacobs, 1986 
Sponsor 


students 


200 


-4.0% 


+ 2.0% 


6.0%R 


short (1 page, 14 items ) 
(vs 2 p, 20 items) 


Erdos & Regier, 1977 


Wall Street Journal 


400 


+ 19.0% 


+ 16.0% 


3.0%D 


familiar sponsor 


Taylor, 1987 


citrus growers 


3,467 


+ 16.9% 


+ 19.4% 


2.5%I 


familiar sponsor 


Appeal 














Roberts etal., 1978 


dentists 


1,190 


-3.0% 


- 2.6% 


0.4%D 


social appeaF- 


Sirken Pifer & Brown, 1960 


families of decedents 


658 


- 18.0% 


- 13.0% 


5.0%D 


help the sponsor 


Other 














Elkind, Try on & deVito, 1986 


psychologists 


500 


+ 2.4% 


- 2.8% 


5.2%R 


university envelope (vs rubber 
stamped) 


Hinrichs, 1975 


administrators 


2,547 


-2.6% 


+ 4.0% 


6.6%R 


with commitment card 


McKee, 1992 


nonprofit prof, 
org. members 


280 


+ 20.0% 


+ 22.8% 


2.8%I 


ID coded & explained 


Roberts, McCrory & 
Forthofer, 1978 


dentists 


1,190 


+ 5.8% 


+ 73% 


1.5%I 


deadline stated 


Wagner & O’Toole, 1985 


psychology dept. heads 106 


-34.0% 


- 70.0% 


36.0%I 


humorous cover letter 
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Table 2 



Change in Treatment Effect After One Followup 



Amount of change 


n 


% 


Less than 1% change 


7 


14 


1.0% -1.9% 


5 


10 


2.0% - 2.9% 


8 


16 


3.0% - 3.9% 


5 


10 


4.0% - 4.9% 


5 


10 


5.0% - 5.9% 


6 


12 


6.0% - 6.9% 


5 


10 


7.0% - 7.9% 


3 


6 


8.0% - 8.9% 


2 


4 


9.0% - 9.9% 


1 


2 


10.0% or more 


3 


6 



When more than one followup contact was made, mixed results were also found, as shown in 
Tables 3 and 4. In many cases, continued followup contacts further reduce the difference between 
groups, particularly through the third contact. While some of the differences may have beensubstantial 
after the initial contact, in some cases they became negligible (3%) after the third contact. James & 
Bolstein (1990) found group differences decreased from 85% after one contact to 0.5% after the third; 
Sirken, Pifer & Brown (1960) saw a decrease from 18% to 2%. Others, however, such as Rucker, Hughes, 
Thompson, Harrison, & Vanderlip (1984) found group differences increased as a result of two additional 
contacts. 

As was true for those cases where information was available after the second contact, over one 
third of the cases (9 or 24) showed differences between groups of less than 3% after the third contact. In 
examining the results in regard to particular independent variables, both studies utilizing appeals to 
promote response (Roberts, McCrory & Forthofer, 1978; Sirken, Pifer & Brown, 1960) found the effects of 
those appeals virtually eliminated after three contacts, and the two studies investigating personalization 
(Green & Kvidahl, 1989; Roberts, McCrory & Forthofer, 1978) found little change in group differences. 
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Table 3 



Group Differences After Multiple Followups 



Variable - Study 


Sample 


N 


di 


d2 




d4 


Condition 


Incentives 
















James & Bolstein, 1990 


cable subscribers 


850 


+ 8.5% 


+ 2.6% 


- 03% 


- 1.7% 


25 incentive 








+ 8.9% 


+ 6.0% 


- 0.6% 


-1.6% 


30 








+18.6% 


+16.2% 


+ 8.4% 


+ 4.8% 


$1.00 








+ 23.4% 


+16.2% 


+ 11.4% 


+ 7.8% 


$2.00 


James & Bolstein, 1992 


subcontractors 


1,200 


+ 20.0% 


+ 153% 


+ 14.6% 


+ 12.0% 


$1 








+ 28.0% 


+ 24.0% 


+ 20.0% 


+ 193% 


$5 cash 








+ 31.3% 


+ 26.0% 


+ 20.0% 


+ 153% 


$5 check 








+ 23.3% 


+ 20.0% 


+ 15.3% 


+ 14.7% 


$10 check 








+ 33.3% 


+ 34.0% 


+ 28.6% 


+ 273% 


$20 check 








+ 33.3% 


+ 26.6% 


+ 19.3% 


+ 173% 


$40 check 








+ 2.6% 


+ 6.6% 


+ 5.6% 


+ 4.7% 


promise of $50 


Sponsor 
















Rucker, Hughes, Thompson, 


alumni 


384 


+ 5.7 




+ 0.2% 




animal science (vs 


Harrison, & Vanderlip, 1984 














textiles & clothing) 


Rucker et al., 1984 


alumni 


384 


+ 1.6% 




+ 0.8% 




student (vs professor) 


Taylor, 1987 


citrus growers 


3,467 


+ 16.9% 


+ 19.4% 


+ 17.3% 




familiar sponsor 


Personalization 
















Green & Kvidahl, 1989 


teachers 


600 


+ 10.0% 




+ 11.0% 




personalized letter. 
















salutation, address, 
signature 


Roberts, McCrory & 


dentists 


1,190 


+ 1.4% 


- 0.9% 


+ 2.4% 




personalized salutation 


Forthofer, 1978 
















Anneal 
















Roberts etal. 1978 


dentists 


1,190 


- 3.0% 


-2.6% 


-1.4% 




social appeal 


Sirken, Pifer & Brown, 1960 


decedent families 


658 


- 18.0% 


-13.0% 


-2.0% 




helpihe sponsor 


Other 

















Anderson, Niebuhr & Gum, 
1987 


doctors 


132 




+ 20.0% 


+ 13.0% 


+ 1.0% 


Boser, 1990 


alumni 


297 


+ 2.4% 


+ 23% 


+ 3.2% 


-2.9% 


Brown, 1965 


physicians 


523 


+ 15.0% 


+ 1.0% 


0.0% 




Roberts etal., 1978 


dentists 


1,190 


+ 5.8% 


+ 73% 


+ 2.6% 




Rucker et al., 1984 


alumni 


384 


+ 1.3% 




-163% 





- .2% - 93% 



outgoing Fed Ex mail 
(vs 1st class) 

booklet format 

length - Postcard (vs 1 page) 

deadline stated 

photo of professionally 
dressed female 
on cover letter 

casually dressed female 
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Table 4 

Effect of Multiple Followups 



Variable - Study Sample N fe @d2 fe@d3 fe@d4 Condition 



Incentives 



James & Bolstein, 1990 


cable subscribers 


850 


5.9%D 


3.196R 


1.2961 


25 incentive 








Z9%D 


6.6%R 


1.0961 


.50 








2.4%D 


7.8%D 


3.6%D 


$1.00 








7.2%D 


4.8%D 


3.6%D 


$2.00 


James & Bolstein, 1992 


subcontractors 


1,200 


4.7%D 


0.7%D 


2.6%D 


$1 








4.0%D 


4.0%D 


0.7%D 


$5 cash 








5.3%D 


6.0%D 


4.7%D 


$5 check 








3.3%D 


4.7%D 


0.6%D 


$10 check 








0.7%I 


5.4%D 


1.396D 


$20 check 








6.7%D 


7.3%D 


Z0%D 


$40 check 








4.0%I 


1.096D 


0.9%D 


promise of $50 


Sponsor 














Rucker, Hughes, Thompson, 


alumni 


384 


♦ 


5.5%D 




animal science (vs 


Harrison, & Vanderlip, 1984 






♦ 






textiles & clothing) 


Rucker et al., 1984 


alumni 


384 




1.696D 




student (vs professor) 


Taylor, 1987 


citrus growers 


3,467 


2.5 %I 


2.196D 




familiar sponsor 


Personalization 














Green & Kvidahl, 1989 


teachers 


600 


♦ 


1.0961 




personalized letter, signature. 














salutation, address, 


Roberts, McCrory & 


dentists 


1,190 


2.3%R 


3.3%R 




personalized salutation 


Forthofer, 1978 














Appeal 














Roberts et al., 1978 


dentists 


1,190 


0.4%D 


1.296D 




social appeal 


Sirken, Pifer & Brown, 1960 


decedent families 


658 


5.0%D 


11.096D 




help the sponsor 


Other 












TTf- 


Anderson, Niebuhr & Gum, 


doctors 


132 


** 


7.0%D 


12.096D 


outgoing Fed Ex mail 


1987 












(vs 1st class) 


Boser, 1990 


alumni 


297 


0.1%I 


0.7961 


6.196R 


booklet format 


Brown, 1965 


physician s 


523 


14.0%D 


1.096D 




length - Postcard (vs 1 page) 


Roberts etal., 1978 


dentists 


1,190 


1.5961 


4.7%D 




deadline stated 


Rucker et al., 1984 


alumni 


- 


♦ 


17.896R- 




photo of professionally 














dressed female 














on cover letter 








♦ 


9.3%I 




casually dressed female 



*No response rates reported after d2 (second followup) 
**No response rates reported after initial contact 
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Regarding the use of incentives, most forms of incentives studied by James & Bolstein (1990,1992) showed 
consistently less impact on response rates as additional followups were instituted. Exceptions were in the 
use of a $20 check and the promise of $50, and cash incentives of less than a dollar. Cash incentives of one 
dollar or more and checks still showed distinct advantages over control groups who did not receive 
incentives even after the fourth contact. - 

Some individual studies dealing with a particular independent variable raise particularly 
interesting questions about the impact of followups. While there was little difference due to the use of 
photos on the cover letter after the initial mailing (Rucker et al., 1984), those receiving the photo version 
were characterized by considerably lower response rates than the control group after the third contact . 
Inconsistent or erratic results were found between contacts in studies concerning familiarity with sponsor 
(Taylor, 1987), questionnaire format (Boser, 1990), use of a deadline (Roberts et al., 1978), and 
personalization (Roberts et al., 1978). 

Conclusions 

While findings tend to be inconclusive, it can be seen that followups have the potential for 
confounding the results of experiments aimed at determining the most effective procedures for 
facilitating response rates in mail surveys. The most consistent findings are that one followup was more 
likely to decrease the effect of the manipulation than to enhance or reverse it, especially the use of 
incentives. However, the magnitude of group differences must be considered along with the effect of the 
additional contact(s). In some cases the difference between groups is lessened, but the group response 
rates are still considerably different, thus the manipulation would still be recommended. 

With the limited number of cases available, it is not possible to draw hard and fast conclusions 
regarding many of the independent variables studied or other study characteristics such as the target 
population, recency of the study, etc. It is to be strongly recommended that reports dealing with 
experimental studies regarding mail survey methods be complete in documenting group response rates 
after each contact so that the reader can fully understand the study and evaluate the findings. 
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